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ABSTRACT 


Two  visually  meaningful  correlation  measures 
are  proposed  for  comparing  calculated  and 
measured  response  histories .  One  is  an  error 
index  which  is  a  simplification  of  RSS 
(root-sum-square)  error  factor,  and  the  other  is 
an  inequality  index  which  is  a  simplification  of 
Theil's  inequality  coefficient.  The  first 
compares  the  difference  between  the  calculated 
and  the  measured  histories  to  the  measured 
history.  The  second  compares  the  difference 
between  the  two  histories  to  the  sum  of  the  two. 
The  proposed  correlation  measures  are  compared  to 
other  existing  measures,  namely,  Geers'  Error 
Factors,  RSS  Error  Factor,  and  Theil's  Inequality 
Coefficient  for  ease  of  interpretation  and 
visualization. 


ADMINISTRATIVE  INFORMATION 

This  work  was  co-sponsored  by  Naval  Sea  Systems  Command  (PMS 
350)  for  the  IPMP-II  SEAWOLF  MPU  (Main  Propulsion  Unit)  Shock 
Qualification:  CSA  (Comparative  Shock  Analysis)  Program  and  by  the 
Office  of  Naval  Research  (ONR  4523)  in  partial  fulfillment  of 
Milestone  1  of  Task  2  of  the  Survivability/Hull  Structures  Project 
(RB23S22)  of  the  Submarine  Technology  Block  Program  (ND3A/PE62323N) , 
and  was  performed  by  the  Submarine  Protection  Department,  Code  67.1 
of  the  Survivability,  Structures  and  Materials  Directorate  of 
Carderock  Division,  Naval  Surface  Warfare  Center. 


INTRODUCTION 

For  over  a  decade,  the  Navy's  shock  community  has  been  using 
Geers'  error  factors  (magnitude,  phase,  and  comprehensive)^  as  a 
comparison  tool  to  judge  the  "goodness"  of  calculated  response 
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histories  against  the  measured.  While  the  Geers'  pioneering  work  has 
served  its  purpose  and  will  continue  to  serve  as  a  comparison  tool,  a 
concern  raised  by  the  senior  author  during  a  recent  SEAWOLF  MPU/CSA 
review  meeting  that  some  of  the  Geers'  error  factors  somehow  "did  not 
look  right"  led  to  this  work. 

In  comparing  any  two  response  histories  (both  calculated,  both 
measured,  or  one  calculated  and  the  other  measured),  a  common 
practice  is  to  assume  one  of  the  two  to  be  "true"  (exact  or  accurate) 
and  the  other  "approximate,"  and  any  discrepancy  or  deviation  from 
the  true  is  associated  with  the  term  "error".  For  example,  when 
calculated  values  are  compared  to  measured  values,  it  may  be 
"scientifically  correct."  to  assume  the  measured  to  be  "true".  Very 
often,  however,  the  measured  values  can  be  as  uncertain  as  the 
calculated  values,  and  thus  there  would  be  no  justification  for 
favoring  the  measured  over  the  calculated,  in  which  case  any 
difference  between  the  two  is  associated  with  the  term  "inequality", 
as  opposed  to  "error". 

To  accommodate  both  of  the  above  cases,  i.e.,  "error"  and 
"inequality",  two  correlation  measures  are  proposed  herein: 

Zilliacus'  error  index  and  Whang's  inequality  index.  The  proposed 
error  index  compares  the  difference  between  two  histories  to  the  one 
assumed  to  be  "true",  and  for  convenience  the  assumed  "true"  values 
will  be  called  the  "measured",  m(t),  and  the  other  the  "calculated", 
c(t).  The  error  index  is  a  simplification  of  the  well-known  RSS 
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(root-sum-square)  error  factor.  The  proposed  inequality  index,  on 
the  other  hand,  compares  the  difference  between  two  histories  to  the 
sum  of  the  two,  without  assuming  one  of  the  two  to  be  "true".  The 
two  histories  can  be  any  combination  of  calculated  and  measured; 
however,  in  this  report,  one  will  be  called  the  calculated,  c(t),  and 
the  other  the  measured,  m(t),  for  convenience.  The  inequality  index 
is  a  simplification  of  Theil's  inequality  coefficient^'^. 

The  report  first  presents  Geers'  error  factors  and  RSS  error 
factor,  followed  by  the  proposed  error  index.  The  report  then 
presents  Theil's  inequality  coefficient  and  the  proposed  inequality 
index.  Discussions  of  each  of  the  correlation  measures  and  their 
comparisons  then  follow. 
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CORRELATION  MEASURES 


In  what  follows,  are  the  calculated  values,  and  are  the 
measured  values . 

GEERS'  Error  Factors  fM,  P.  C^t 

a )  Magnitude  Error  Factor 


IlA 

M  =  - ,  -  1  (1) 

JY< 


Since  the  first  term  can  be  less  than  1,  M  can  be  negative,  and 
since  the  first  term  can  be  greater  than  2,  M  can  exceed  1. 

b)  Phase  Error  Factor  (P^t 

/lZ=i"il 

P  =  1 -  -  -  —  (2) 

Since  the  second  term  cannot  exceed  1,  P  is  bounded  between  0 
and  1 . 

c )  Comprehensive  Error  Factor  (C^; 

C  =  J  +  P^  (3) 

C  is  the  vectorial  sum  of  its  orthogonal  components  M  and  P,  and 
since  M  can  exceed  1,  C  can  exceed  1, 
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R  is  the  RSS  of  the  differences  between  and  divided  by  the 
RSS  of  m .  Obviously,  R  can  exceed  1. 


Z  is  the  area  of  the  residual  (Cj^-irij^)  divided  by  the  area  of  the 
measured,  and  it  can  exceed  1. 


T  is  the  RSS  of  the  differences  between  Cj^  and  mj^  divided  by  the 
sum  of  the  RSS  of  Cj^  and  the  RSS  of  m^^ .  T  is  bounded  between  0  and  1. 


Z  l^i|  Z 


W  is  the  area  of  the  residual  {c^^-mj^)  divided  by  the  sum  of  the 
areas  of  the  calculated  and  measured.  W  is  bounded  between  0  and  1. 
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DISCUSSION 


Figures  1(a),  2(a),  and  3(a)  are  directly  from  Reference  1,  and 
they  indicate  Geers'  Magnitude  (6^),  Phase  (ep),  and  Comprehensive 
(e^^)  error  factors  along  with  RSS  (e^.)  error  factor  for  three 
different  sets  of  comparisons.  The  starting  times  for  the  calculated 
c(t)  and  the  measured  m(t)  are  different  because  Geers'  error 
measures  require  m(t)  (or  c(t))  to  be  adjusted  horizontally  until  e^ 
is  minimized.  This  adjustment  was  deemed  necessary  in  fairness  to 
the  analysts  because  the  starting  time  (t=0)  for  m(t)  is  usually  not 
well  known.  In  the  future,  before  any  comparison  is  made,  a  common 
reference  time,  such  as  the  time  from  detonation,  should  be  clearly 
indicated  on  each  history  (calculated  and  measured)  so  that  the 
adjustment  of  starting  times  would  not  be  necessary. 

Figures  1(b),  2(b),  and  3(b)  correspond  to  their  respective 
Figures  without  time  adjustments,  and  in  each  Figure,  M,  P,  C,  and  R 
are  shown.  While  C's  are  greater  than  e^'s,  the  differences  are  not 
drastic.  In  Reference  1,  Geers  points  out  that  the  e^-'s  in  Figures 
2(a)  and  3(a)  seem  to  be  unacceptably/too  large.  However,  since  the 
upper  bound  of  e^.  is  limitless  (not  bounded  by  100%)  -here  is  no 
basis  for  saying  these  values  appear  too  large.  In  fact,  e^  is  not 
bounded  either.  At  least,  ej.=100%  can  be  understood  to  be  the  case 
when  the  RSS  of  mj^  is  equal  to  the  RSS  of  (Cj^-mj^),  though  not 
easily  visualized.  The  case  where  ej,=100%  is  beyond  visualization. 
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The  most  troublesome  of  these  Figures  is  Figure  2  which  shows 
ejjj=0%  (or  M=0.009)  and  ej,=4%  (or  C=0.139).  Most  observers  would 
agree  that  the  magnitude  errors  of  these  sets  are  nowhere  near  zero. 

In  Figure  4,  the  previous  Figures  1(b),  2(b)  and  3(b)  are 
repeated,  and  Z,  T,  and  W  are  added  for  comparison.  Z  and  R  are 
consistently  similar  to  each  other  as  one  might  expect,  but  the 
advantage  of  Z  over  R  is  that  Z  is  easier  to  visualize  than  R.  Both 
Z  and  R  can  be  greater  than  1.  As  pointed  out  earlier,  Z  can  be 
visualized  as  the  ratio  of  the  area  of  the  residual  (Cj^-mj^)  to  the 
area  of  the  measured.  (See  Figures  A-1  through  A-3  in  Appendix  A.) 

The  difference  between  R  and  T  is  that  the  denominator  of  T  has 
an  additional  term,  the  square  root  of  the  sum  of  the  squares  of  Cj^ . 
Conceptually,  T  compares  the  RSS  of  the  residuals  to  the  sum  of  the 
RSS's  of  the  measured  and  the  calculated;  therefore,  when  (Cj^-mj^)  are 
small,  T  tends  to  be  about  half  of  R.  T  can  never  be  greater  than  1, 
while  R  is  unbounded. 

The  difference  between  Z  and  W  is  that  the  denominator  of  W  has 
an  additional  term,  the  sum  of  the  absolute  values  of  Cj^;  therefore, 
when  (c^-mj^)  are  small,  W  tends  to  be  about  half  of  Z.  W  can  never 
be  greater  than  1,  while  Z  is  unbounded. 


7 


T  and  W  are  consistently  similar  to  each  other  as  one  might 
expect,  but  the  advantage  of  W  over  T  is  that  W  is  easier  to 
visualize  than  T.  As  pointed  out  earlier,  W  can  be  visualized  as  the 
ratio  of  the  area  of  the  residual  (Cj^-mj^)  to  the  sum  of  the  areas  of 
the  calculated  and  the  measured.  (See  Figures  A-1  through  A- 3  in 
Appendix  A. ) 

Figures  5  and  6  show  what  happens  when  the  sign  of  one  of  the 
two  curves  is  reversed.  As  pointed  out  by  Dawson^,  M,  P,  and  C  are 
insensitive  to  the  sign  reversal.  {Also,  note  that  in  these  cases  R 
and  Z  are  greater  than  1,  while  T  and  W  are  not.)  Recently,  to 
remedy  this  problem,  Geers^  proposed  a  revision  to  P  which  in  turn 
affects  C.  The  new  Geers'  error  factors  are  as  follows:  (K  is 
unchanged . ) 


^  c  .m . 


Since  the  second  term  in  Pnew  is  bounded  between  +1  and  -1,  Pnew  is 
now  bounded  between  0  and  2.  Cnew  can  still  exceed  1.  As  shown  in 
Fig..r-  and  6(b),  Pnew  and  therefore  Cnew  are  sensitive  to  the 

sign  reversal.  In  fact,  Pnew=2.0  is  an  indication  of  sign  reversal 
or  of  being  completely  out  of  phase. 


The  fact  that  Z  can  exceed  1  is  meaningful  since  the  "residual" 
can  be  greater  than  the  "measured".  Similarly,  the  case  of  W=1.0  is 
meaningful  since  that  happens  when  the  "residual"  is  equal  to  the 
"sum".  However,  since  C(or  Cnew)  is  the  vectorial  sum  of  M  and  P(or 
Pnew),  the  meaning  of  C(or  Cnew)=1.0  is  not  clear. 

Figure  7  shows  the  effect  of  reversing  c(t)  and  m(t).  M  is 
sensitive  to  the  reversal  as  pointed  out  by  Dawson^,  while  P  and  Pnew 
are  not.  Also,  as  expected,  R  and  Z  are  sensitive  to  the  reversal, 
while  T  and  W  are  not. 

Figure  8  is  not  an  example  of  common  occurrence  but  is  presented 
here  to  show  the  need  to  distinguish  between  "early"  time  comparison 
and  "late"  time  comparison.  The  integration  limits  have  been 
arbitrarily  selected  as  0  to  1.0,  1.0  to  2.0,  and  0  to  2.0. 

Figures  9,  10,  11  are  related  to  a  common  case  involving  strain 
records  showing  a  permanent  set.  Figure  9(a)  shows  that  when  c(t) 
and  m(t)  are  very  close  to  each  other,  all  of  the  measures  are 
reasonable.  In  Figure  9(b),  R,  Z,  T,  and  W  appear  to  be  reasonable, 
while  M,  P,  C,  Pnew  and  Cnew  appear  to  be  a  bit  low.  Figure  10  is  an 
example  of  the  cases  where  M,  C,  Cnew,  R,  and  Z  are  close  to  each 
other  and  where  T  and  W  are  not  about  half  of  R  and  Z,  respectively. 
Figure  11  shows  the  cases  where  c(t)  shows  no  sign  of  permanent  set. 
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Figure  12  is  presented  here  to  show  that  when  W=1.0,  i.e., 
when  t-he  "difference"  is  equal  to  the  "sum",  the  measure  does  not 
distinguish  the  degree  of  badness,  i.e.,  W  indicates  that  the  Figures 
12(a)  and  12(b)  are  "equally  bad".  On  the  other  hand,  C  values  show 
Figure  12(b)  to  be  worse  than  Figure  12(a),  while  R  and  Z  values  show 
the  reverse.  Pnew  does  not  distinguish  the  degree  of  badness  when 
two  curves  are  completely  out  of  phase. 

The  error  measures  and  inequality  measures  discussed  above  are 
summarized  on  the  next  page  for  the  reader's  convenience. 


10 


VISUALIZATION 


For  a  correlation  measure  to  be  visually  meaningful,  it  must  be 
simple  and  consistent  with  "eyeballing",  i.e.,  the  mental  process  of 
human  eyes  to  compare  two  response  histories.  One  possible  process 
is  to  pick  off  a  vertical  distance  (absolute  value)  of  the  difference 
between  two  curves  at  a  particular  time  and  divide  that  distance  by 
another  vertical  distance  at  that  particular  time.  These  ratios  can 
be  obtained  at  various  times,  and  with  relative  ease,  they  can  be 
averaged  mentally  in  an  approximate  way. 

The  above  process  is  expressed  mathematically  as  follows: 


'A'  resembles  Z,  while  'B'  resembles  W.  'A'  can  be  visualized  by 
most  people,  but  can  'B'  be  visualized?  To  answer  that  question,  an 
unscientific/informal  survey  was  taken  among  a  number  of  colleagues 
at  Carderock  Division.  The  result  of  the  survey  is  tabulated  in 
Figure  13.  For  each  s  -t  of  curves,  three  numbers  were  given,  and  the 
participants  were  asked  to  estimate  the  value  of  'B'  visually,  and 
select  one  number  closest  to  that  value.  The  percentages  for  each 
set  show  the  result  of  the  survey.  For  example.  Figure  13(b)  shows 
that  17%  of  the  participants  selected  0.770  to  be  the  value  of  'B', 
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while  75%  thought  that  0.388  was  the  value  of  'B'.  The  letters  in 
parentheses  next  to  "%"  indicate  which  values  they  actually  were.  In 
each  of  these  cases,  most  of  the  participants  picked  W  values  to  be 
the  'B'  values,  demonstrating  that  the  'B'  values  can  be  visualized 
and  that  they  are  similar  to  W  values.  However,  the  problem  with  'A' 
and  'B'  is  that  when  c(t)  and  m(t)  intersect  simultaneously  on  the 
time  axis,  i.e.,  when  |cj^-mjLl=0  and  |mj^l=0  or  when  |c£-m£|=0  and 
I  C£  I  +  I  m^^  I  =0 ,  the  condition  of  indeterminacy  occurs.  (See  Figure 
4(a),  for  example.)  This  problem  has  been  avoided  in  Z  and  W  by 
summing  in  the  numerator  and  in  the  denominator  separately.  This 
enables  one  to  associate  Z  and  W  as  the  ratios  of  areas.  It  should 
be  pointed  out  here  that  visualizing  or  mentally  estimating  M,  P,  C, 
Pnew,  Cnew,  R,  and  T  is  not  natural  for  most  people  because  it 
involves  taking  the  square  root  of  the  sum  of  squares  of  many 
numbers . 

In  Reference  5,  Geers  used  a  simple  case  of  Cj^=kmj^,  where  k  is  a 
constant,  to  point  out  correctly  that  W  is  not  symmetric  about  k=l.* 
For  example,  if  one  analysis  produced  C£=0.5mj^,  W  would  be  0.333 
(=0.5/1.5),  while  if  another  analysis  produced  Cj^^^l.Smj^,  its  W  would 
be  0.200  (=0. 5/2.5).  And  since  each  differed  from  the  measured  by 
0 . 5m£ ,  Geers  states  that  associating  two  different  values  of  W  for 
these  two  analyses  is  "not  proper".  It  is  true  that  W  is  not 


*  In  this  example, 

M=|k|-1,  P=0,  C=||k|-l|,  R=Z=|k-l|,  and  T=W= | k-1 1 / ( | k | +1 ) . ^ 
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symmetric  about  k=l  as  shown  in  Figure  14  for  this  simple  case.  (C 
is  locally  symmetric  about  k=l,  and  globally  syiranetric  about  k=0. 

Z  is  symmetric  about  k=l.)  Howeyer,  the  way  human  eyes  compare  two 
curves  is  not  symmetric  about  k=l.  Using  the  Geers'  example  of 
Cj^^=kmj^,  Figure  15  shows  the  asymmetric  nature  of  "eyeballing". 

Figure  15(a)  has  the  appearance  of  better  correlation  than  Figure 
15(b),  eyen  though  both  c(t)'s  differ  from  m(t)  by  50% !  Figures 
15(c)  and  15(d)  make  the  point  more  strikingly. 

Figure  16  shows  further  the  asymmetric  nature  of  "eyeballing" 
for  exponentially  decaying  sine  wayes  for  k  values  of  0.5,  1.5,  2.0 
and  3.0.  Figure  16(b)  appears  to  have  better  correlation  than  Figure 
16(a),  even  though  they  both  differ  from  the  measured  by  50%  as  shown 
by  C=Z=0.500  in  both  cases.  (In  this  example,  P=Pnew=0;  therefore 
C=Cnew. )  Figures  16(a)  and  16(c)  have  the  same  W,  but  they  have 
different  C  and  Z.  Figures  16(c)  and  16(d)  show  that  Z=1.0  in 
general  means  the  "calculated"  on  the  whole  is  twice  the  "measured" 
and  that  Z=2.0  means  the  "calculated"  on  the  whole  is  three  times  the 
"measured",  etc.  (See  Figures  A-15  through  A-18  in  Appendix  A.) 

To  complete  the  discussion  on  visualization.  Appendix  A  is 
provided  to  show  how  Z  and  W  can  be  interpreted.  In  each  Figure, 
under  the  original  c(t)  and  m(t)  (with  a  reference  to  the  original 
Figure  number),  the  rectified  (absolute-valued)  residual  [c(t)-m(t)] 
is  compared  to  the  rectified  m(t)  for  Z,  and  below  that,  the 
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rectified  residual  is  compared  to  the  svim  of  the  rectified  c(t)  and 
m(t)  for  W.  In  almost  all  of  the  cases  shown,  the  Z-curves 
intersect,  while  in  all  cases,  the  W-curves  never  intersect,  although 
at  some  points  they  are  coincident,  which  happens  when  the 
"difference"  is  equal  to  the  "sum". 

One  might  recognize  that  both  Z  and  W  are  the  ratios  of  means: 

Z  is  the  ratio  of  the  mean  of  the  rectified  residual  (shaded  area)  to 
the  mean  of  the  rectified  measured,  and  W  is  the  ratio  of  the  mean  of 
the  rectified  residual  (shaded  area)  to  the  mean  of  the  sum  of  the 
rectified  calculated  and  the  rectified  measured*.  However,  visually 
comparing  the  areas  is  more  appealing  than  averaging.  After  all, 
visually  comparing  the  areas  is  the  whole  idea  behind  "pie  charts". 

It  is  not  necessary  to  compare  every  experimental  record  to 
analysis  in  this  graphical  fashion.  These  graphs  are  presented  here 
to  show  that  Z  and  W  are  consistent  with  the  way  one  might  visualize 
or  interpret  these  values . 

Several  computer  programs  (ERROR,  EQGEN  and  RECTIFY)  were 
written  to  generate  these  curves  and  to  calculate  their  corresponding 
M,  P,  C,  Pnew,  Cnew,  R,  Z,  T,  and  W  values.  The  programs  are 

*  Comparing  Equations  (5)  and  (7),  Z  and  W  can  be  mathematically  combined  in 
the  form  of  (p  +  q)  2 1  -  m^  |  /  (p2|c^|  +  qilmj^l)  where  p  and  q  may  be 

interpreted  as  weighting  factors  or  some  probability  coefficients  related  to 
uncertainties  of  c(t)  and  m(t).  However,  by  doing  so,  the  simplicity  and 
the  visual  meaning  of  Z  and  W  would  be  lost. 
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available  upon  request,  and  even  if  they  are  not  requested,  they  are 
still  available. 


CONCLUSIONS  AND  RECOMMENDATIONS 


Two  visually  meaningful  correlation  measures  have  been  proposed 
for  comparing  "calculated"  and  "measured"  response  histories; 
Zilliacus'  error  index  and  Whang's  inequality  index.  The  error  index 
(Z)  is  appropriate  when  there  is  justification  for  favoring  the 
"measured"  over  the  "calculated".  The  inequality  index  (W)  is 
appropriate  when  there  is  no  justification  for  favoring  one  over  the 
other.  However,  whether  there  is  justification  or  not,  both  of  the 
proposed  measures,  Z  and  W,  may  be  used  to  supplement  Geers'  Cnew 
without  adjusting  starting  times  as  long  as  what  they  are  comparing 
and  what  they  are  comparing  to  are  kept  in  mind. 

As  the  community  gains  experience  in  associating  the  values  of 
Cnew,  Z,  and  W  with  their  corresponding  plots,  a  consensus  may  be 
reached  on  qualitative  words  to  go  with  certain  ranges  of  these 
values.  For  example,  "Excellent"  may  be  assigned  for  W  values  less 
than  0.1,  "Good"  to  "Fair"  for  W  values  between  0.1  and  0.4,  etc., 
and  similar  words  for  Cnew  and  Z.  These  words  are  clearly 
subjective,  and  the  range  of  values  for  each  word  would  change  as 
both  the  computational  and  experimental  technologies  improve.  In  the 
meantime,  it  is  hoped  that  the  proposed  correlation  measures  would 
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help  in  making  the  subjective  judgment  of  whether  an  analysis  on  the 
whole  is  acceptable  or  not  acceptable. 
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(See  Figure  4(a)) 
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(See  Figure  5(b)) 


39 


(See  Figure  7(c)) 
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